

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>prepare &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/jquery.js"></script>
        <script src="../../../_static/underscore.js"></script>
        <script src="../../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../../genindex/" />
    <link rel="search" title="Search" href="../../../search/" />
    <link rel="next" title="create" href="../create/" />
    <link rel="prev" title="加密" href="../encryption/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../../">ceph-volume</a> &raquo;</li>
        
      <li><code class="docutils literal notranslate"><span class="pre">prepare</span></code></li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../../_sources/ceph-volume/lvm/prepare.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../../">
          

          
            
            <img src="../../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../">ceph-volume</a><ul class="current">
<li class="toctree-l2 current"><a class="reference internal" href="../../#id2">迁移</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../../#id3">全新部署</a></li>
<li class="toctree-l3 current"><a class="reference internal" href="../../#osd">已有 OSD 怎么办</a><ul class="current">
<li class="toctree-l4"><a class="reference internal" href="../../intro/">Overview</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../intro/#replacing-ceph-disk">Replacing <code class="docutils literal notranslate"><span class="pre">ceph-disk</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../intro/#gpt-partitions-are-simple">GPT partitions are simple?</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../intro/#modularity">Modularity</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../intro/#ceph-volume-lvm"><code class="docutils literal notranslate"><span class="pre">ceph-volume</span> <span class="pre">lvm</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../intro/#lvm-performance-penalty">LVM performance penalty</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../inventory/"><code class="docutils literal notranslate"><span class="pre">inventory</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../drive-group/"><code class="docutils literal notranslate"><span class="pre">drive-group</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../"><code class="docutils literal notranslate"><span class="pre">lvm</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../activate/"><code class="docutils literal notranslate"><span class="pre">activate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../batch/"><code class="docutils literal notranslate"><span class="pre">batch</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../batch/#ceph-volume-lvm-batch-report">报表</a></li>
<li class="toctree-l4"><a class="reference internal" href="../batch/#sizing">Sizing</a></li>
<li class="toctree-l4"><a class="reference internal" href="../batch/#idempotency-and-disk-replacements">Idempotency and disk replacements</a></li>
<li class="toctree-l4"><a class="reference internal" href="../encryption/">加密</a></li>
<li class="toctree-l4 current"><a class="current reference internal" href="#"><code class="docutils literal notranslate"><span class="pre">prepare</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../create/"><code class="docutils literal notranslate"><span class="pre">create</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../scan/">scan</a></li>
<li class="toctree-l4"><a class="reference internal" href="../systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../list/"><code class="docutils literal notranslate"><span class="pre">list</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../zap/"><code class="docutils literal notranslate"><span class="pre">zap</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../migrate/"><code class="docutils literal notranslate"><span class="pre">migrate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../newdb/"><code class="docutils literal notranslate"><span class="pre">new-db</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../newwal/"><code class="docutils literal notranslate"><span class="pre">new-wal</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../simple/"><code class="docutils literal notranslate"><span class="pre">simple</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../simple/activate/"><code class="docutils literal notranslate"><span class="pre">activate</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../simple/scan/"><code class="docutils literal notranslate"><span class="pre">scan</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../simple/systemd/">systemd</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../zfs/"><code class="docutils literal notranslate"><span class="pre">zfs</span></code></a></li>
<li class="toctree-l4"><a class="reference internal" href="../../zfs/inventory/"><code class="docutils literal notranslate"><span class="pre">inventory</span></code></a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="prepare">
<span id="ceph-volume-lvm-prepare"></span><h1><code class="docutils literal notranslate"><span class="pre">prepare</span></code><a class="headerlink" href="#prepare" title="Permalink to this headline">¶</a></h1>
<p>This subcommand allows a <a class="reference internal" href="../../../glossary/#term-filestore"><span class="xref std std-term">filestore</span></a> or <a class="reference internal" href="../../../glossary/#term-bluestore"><span class="xref std std-term">bluestore</span></a> setup. It is
recommended to pre-provision a logical volume before using it with
<code class="docutils literal notranslate"><span class="pre">ceph-volume</span> <span class="pre">lvm</span></code>.</p>
<p>Logical volumes are not altered except for adding extra metadata.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>This is part of a two step process to deploy an OSD. If looking for
a single-call way, please see <a class="reference internal" href="../create/#ceph-volume-lvm-create"><span class="std std-ref">create</span></a></p>
</div>
<p>To help identify volumes, the process of preparing a volume (or volumes) to
work with Ceph, the tool will assign a few pieces of metadata information using
<a class="reference internal" href="../../../glossary/#term-LVM-tags"><span class="xref std std-term">LVM tags</span></a>.</p>
<p><a class="reference internal" href="../../../glossary/#term-LVM-tags"><span class="xref std std-term">LVM tags</span></a> makes volumes easy to discover later, and help identify them as
part of a Ceph system, and what role they have (journal, filestore, bluestore,
etc…)</p>
<p>Although <a class="reference internal" href="../../../glossary/#term-bluestore"><span class="xref std std-term">bluestore</span></a> is the default, the back end can be specified with:</p>
<ul class="simple">
<li><p><a class="reference internal" href="#ceph-volume-lvm-prepare-filestore"><span class="std std-ref">–filestore</span></a></p></li>
<li><p><a class="reference internal" href="#ceph-volume-lvm-prepare-bluestore"><span class="std std-ref">–bluestore</span></a></p></li>
</ul>
<div class="section" id="bluestore">
<span id="ceph-volume-lvm-prepare-bluestore"></span><h2><code class="docutils literal notranslate"><span class="pre">bluestore</span></code><a class="headerlink" href="#bluestore" title="Permalink to this headline">¶</a></h2>
<p>The <a class="reference internal" href="../../../glossary/#term-bluestore"><span class="xref std std-term">bluestore</span></a> objectstore is the default for new OSDs. It offers a bit
more flexibility for devices compared to <a class="reference internal" href="../../../glossary/#term-filestore"><span class="xref std std-term">filestore</span></a>.
Bluestore supports the following configurations:</p>
<ul class="simple">
<li><p>A block device, a block.wal, and a block.db device</p></li>
<li><p>A block device and a block.wal device</p></li>
<li><p>A block device and a block.db device</p></li>
<li><p>A single block device</p></li>
</ul>
<p>The bluestore subcommand accepts physical block devices, partitions on
physical block devices or logical volumes as arguments for the various device parameters
If a physical device is provided, a logical volume will be created. A volume group will
either be created or reused it its name begins with <code class="docutils literal notranslate"><span class="pre">ceph</span></code>.
This allows a simpler approach at using LVM but at the cost of flexibility:
there are no options or configurations to change how the LV is created.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">block</span></code> is specified with the <code class="docutils literal notranslate"><span class="pre">--data</span></code> flag, and in its simplest use
case it looks like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">bluestore</span> <span class="o">--</span><span class="n">data</span> <span class="n">vg</span><span class="o">/</span><span class="n">lv</span>
</pre></div>
</div>
<p>A raw device can be specified in the same way:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">bluestore</span> <span class="o">--</span><span class="n">data</span> <span class="o">/</span><span class="n">path</span><span class="o">/</span><span class="n">to</span><span class="o">/</span><span class="n">device</span>
</pre></div>
</div>
<p>For enabling <a class="reference internal" href="../encryption/#ceph-volume-lvm-encryption"><span class="std std-ref">encryption</span></a>, the <code class="docutils literal notranslate"><span class="pre">--dmcrypt</span></code> flag is required:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">bluestore</span> <span class="o">--</span><span class="n">dmcrypt</span> <span class="o">--</span><span class="n">data</span> <span class="n">vg</span><span class="o">/</span><span class="n">lv</span>
</pre></div>
</div>
<p>If a <code class="docutils literal notranslate"><span class="pre">block.db</span></code> or a <code class="docutils literal notranslate"><span class="pre">block.wal</span></code> is needed (they are optional for
bluestore) they can be specified with <code class="docutils literal notranslate"><span class="pre">--block.db</span></code> and <code class="docutils literal notranslate"><span class="pre">--block.wal</span></code>
accordingly. These can be a physical device, a partition  or
a logical volume.</p>
<p>For both <code class="docutils literal notranslate"><span class="pre">block.db</span></code> and <code class="docutils literal notranslate"><span class="pre">block.wal</span></code> partitions aren’t made logical volumes
because they can be used as-is.</p>
<p>While creating the OSD directory, the process will use a <code class="docutils literal notranslate"><span class="pre">tmpfs</span></code> mount to
place all the files needed for the OSD. These files are initially created by
<code class="docutils literal notranslate"><span class="pre">ceph-osd</span> <span class="pre">--mkfs</span></code> and are fully ephemeral.</p>
<p>A symlink is always created for the <code class="docutils literal notranslate"><span class="pre">block</span></code> device, and optionally for
<code class="docutils literal notranslate"><span class="pre">block.db</span></code> and <code class="docutils literal notranslate"><span class="pre">block.wal</span></code>. For a cluster with a default name, and an OSD
id of 0, the directory could look like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># ls -l /var/lib/ceph/osd/ceph-0</span>
<span class="n">lrwxrwxrwx</span><span class="o">.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">93</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">block</span> <span class="o">-&gt;</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">ceph</span><span class="o">-</span><span class="n">be2b6fbd</span><span class="o">-</span><span class="n">bcf2</span><span class="o">-</span><span class="mi">4</span><span class="n">c51</span><span class="o">-</span><span class="n">b35d</span><span class="o">-</span><span class="n">a35a162a02f0</span><span class="o">/</span><span class="n">osd</span><span class="o">-</span><span class="n">block</span><span class="o">-</span><span class="mi">25</span><span class="n">cf0a05</span><span class="o">-</span><span class="mi">2</span><span class="n">bc6</span><span class="o">-</span><span class="mi">44</span><span class="n">ef</span><span class="o">-</span><span class="mi">9137</span><span class="o">-</span><span class="mi">79</span><span class="n">d65bd7ad62</span>
<span class="n">lrwxrwxrwx</span><span class="o">.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">93</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">block</span><span class="o">.</span><span class="n">db</span> <span class="o">-&gt;</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sda1</span>
<span class="n">lrwxrwxrwx</span><span class="o">.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">93</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">block</span><span class="o">.</span><span class="n">wal</span> <span class="o">-&gt;</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">-</span><span class="n">wal</span><span class="o">-</span><span class="mi">0</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">37</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">ceph_fsid</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">37</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">fsid</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">55</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">keyring</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span>  <span class="mi">6</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">ready</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span> <span class="mi">10</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="nb">type</span>
<span class="o">-</span><span class="n">rw</span><span class="o">-------.</span> <span class="mi">1</span> <span class="n">ceph</span> <span class="n">ceph</span>  <span class="mi">2</span> <span class="n">Oct</span> <span class="mi">20</span> <span class="mi">13</span><span class="p">:</span><span class="mi">05</span> <span class="n">whoami</span>
</pre></div>
</div>
<p>In the above case, a device was used for <code class="docutils literal notranslate"><span class="pre">block</span></code> so <code class="docutils literal notranslate"><span class="pre">ceph-volume</span></code> create
a volume group and a logical volume using the following convention:</p>
<ul class="simple">
<li><p>volume group name: <code class="docutils literal notranslate"><span class="pre">ceph-{cluster</span> <span class="pre">fsid}</span></code> or if the vg exists already
<code class="docutils literal notranslate"><span class="pre">ceph-{random</span> <span class="pre">uuid}</span></code></p></li>
<li><p>logical volume name: <code class="docutils literal notranslate"><span class="pre">osd-block-{osd_fsid}</span></code></p></li>
</ul>
</div>
<div class="section" id="filestore">
<span id="ceph-volume-lvm-prepare-filestore"></span><h2><code class="docutils literal notranslate"><span class="pre">filestore</span></code><a class="headerlink" href="#filestore" title="Permalink to this headline">¶</a></h2>
<p>This is the OSD backend that allows preparation of logical volumes for
a <a class="reference internal" href="../../../glossary/#term-filestore"><span class="xref std std-term">filestore</span></a> objectstore OSD.</p>
<p>It can use a logical volume for the OSD data and a physical device, a partition
or logical volume for the journal. A physical device will have a logical volume
created on it. A volume group will either be created or reused it its name begins
with <code class="docutils literal notranslate"><span class="pre">ceph</span></code>.  No special preparation is needed for these volumes other than
following the minimum size requirements for data and journal.</p>
<p>The CLI call looks like this of a basic standalone filestore OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">data</span> <span class="o">&lt;</span><span class="n">data</span> <span class="n">block</span> <span class="n">device</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>To deploy file store with an external journal:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">data</span> <span class="o">&lt;</span><span class="n">data</span> <span class="n">block</span> <span class="n">device</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">journal</span> <span class="o">&lt;</span><span class="n">journal</span> <span class="n">block</span> <span class="n">device</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>For enabling <a class="reference internal" href="../encryption/#ceph-volume-lvm-encryption"><span class="std std-ref">encryption</span></a>, the <code class="docutils literal notranslate"><span class="pre">--dmcrypt</span></code> flag is required:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">dmcrypt</span> <span class="o">--</span><span class="n">data</span> <span class="o">&lt;</span><span class="n">data</span> <span class="n">block</span> <span class="n">device</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">journal</span> <span class="o">&lt;</span><span class="n">journal</span> <span class="n">block</span> <span class="n">device</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Both the journal and data block device can take three forms:</p>
<ul class="simple">
<li><p>a physical block device</p></li>
<li><p>a partition on a physical block device</p></li>
<li><p>a logical volume</p></li>
</ul>
<p>When using logical volumes the value <em>must</em> be of the format
<code class="docutils literal notranslate"><span class="pre">volume_group/logical_volume</span></code>. Since logical volume names
are not enforced for uniqueness, this prevents accidentally
choosing the wrong volume.</p>
<p>When using a partition, it <em>must</em> contain a <code class="docutils literal notranslate"><span class="pre">PARTUUID</span></code>, that can be
discovered by <code class="docutils literal notranslate"><span class="pre">blkid</span></code>. THis ensure it can later be identified correctly
regardless of the device name (or path).</p>
<p>For example: passing a logical volume for data and a partition <code class="docutils literal notranslate"><span class="pre">/dev/sdc1</span></code> for
the journal:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">data</span> <span class="n">volume_group</span><span class="o">/</span><span class="n">lv_name</span> <span class="o">--</span><span class="n">journal</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sdc1</span>
</pre></div>
</div>
<p>Passing a bare device for data and a logical volume ias the journal:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">data</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sdc</span> <span class="o">--</span><span class="n">journal</span> <span class="n">volume_group</span><span class="o">/</span><span class="n">journal_lv</span>
</pre></div>
</div>
<p>A generated uuid is used to ask the cluster for a new OSD. These two pieces are
crucial for identifying an OSD and will later be used throughout the
<a class="reference internal" href="../activate/#ceph-volume-lvm-activate"><span class="std std-ref">activate</span></a> process.</p>
<p>The OSD data directory is created using the following convention:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>At this point the data volume is mounted at this location, and the journal
volume is linked:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ln</span> <span class="o">-</span><span class="n">s</span> <span class="o">/</span><span class="n">path</span><span class="o">/</span><span class="n">to</span><span class="o">/</span><span class="n">journal</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster_name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span><span class="o">-</span><span class="nb">id</span><span class="o">&gt;/</span><span class="n">journal</span>
</pre></div>
</div>
<p>The monmap is fetched using the bootstrap key from the OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="nb">bin</span><span class="o">/</span><span class="n">ceph</span> <span class="o">--</span><span class="n">cluster</span> <span class="n">ceph</span> <span class="o">--</span><span class="n">name</span> <span class="n">client</span><span class="o">.</span><span class="n">bootstrap</span><span class="o">-</span><span class="n">osd</span>
<span class="o">--</span><span class="n">keyring</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">bootstrap</span><span class="o">-</span><span class="n">osd</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">mon</span> <span class="n">getmap</span> <span class="o">-</span><span class="n">o</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;/</span><span class="n">activate</span><span class="o">.</span><span class="n">monmap</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code> will be called to populate the OSD directory, that is already
mounted, re-using all the pieces of information from the initial steps:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">osd</span> <span class="o">--</span><span class="n">cluster</span> <span class="n">ceph</span> <span class="o">--</span><span class="n">mkfs</span> <span class="o">--</span><span class="n">mkkey</span> <span class="o">-</span><span class="n">i</span> <span class="o">&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;</span> \
<span class="o">--</span><span class="n">monmap</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;/</span><span class="n">activate</span><span class="o">.</span><span class="n">monmap</span> <span class="o">--</span><span class="n">osd</span><span class="o">-</span><span class="n">data</span> \
<span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">osd</span><span class="o">-</span><span class="n">journal</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;/</span><span class="n">journal</span> \
<span class="o">--</span><span class="n">osd</span><span class="o">-</span><span class="n">uuid</span> <span class="o">&lt;</span><span class="n">osd</span> <span class="n">uuid</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">keyring</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;/</span><span class="n">keyring</span> \
<span class="o">--</span><span class="n">setuser</span> <span class="n">ceph</span> <span class="o">--</span><span class="n">setgroup</span> <span class="n">ceph</span>
</pre></div>
</div>
</div>
<div class="section" id="ceph-volume-lvm-partitions">
<span id="id1"></span><h2>分区<a class="headerlink" href="#ceph-volume-lvm-partitions" title="Permalink to this headline">¶</a></h2>
<p><code class="docutils literal notranslate"><span class="pre">ceph-volume</span> <span class="pre">lvm</span></code> does not currently create partitions from a whole device.
If using device partitions the only requirement is that they contain the
<code class="docutils literal notranslate"><span class="pre">PARTUUID</span></code> and that it is discoverable by <code class="docutils literal notranslate"><span class="pre">blkid</span></code>. Both <code class="docutils literal notranslate"><span class="pre">fdisk</span></code> and
<code class="docutils literal notranslate"><span class="pre">parted</span></code> will create that automatically for a new partition.</p>
<p>For example, using a new, unformatted drive (<code class="docutils literal notranslate"><span class="pre">/dev/sdd</span></code> in this case) we can
use <code class="docutils literal notranslate"><span class="pre">parted</span></code> to create a new partition. First we list the device
information:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ parted --script /dev/sdd print
Model: VBOX HARDDISK (scsi)
Disk /dev/sdd: 11.5GB
Sector size (logical/physical): 512B/512B
Disk Flags:
</pre></div>
</div>
<p>This device is not even labeled yet, so we can use <code class="docutils literal notranslate"><span class="pre">parted</span></code> to create
a <code class="docutils literal notranslate"><span class="pre">gpt</span></code> label before we create a partition, and verify again with <code class="docutils literal notranslate"><span class="pre">parted</span>
<span class="pre">print</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ parted --script /dev/sdd mklabel gpt
$ parted --script /dev/sdd print
Model: VBOX HARDDISK (scsi)
Disk /dev/sdd: 11.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
</pre></div>
</div>
<p>Now lets create a single partition, and verify later if <code class="docutils literal notranslate"><span class="pre">blkid</span></code> can find
a <code class="docutils literal notranslate"><span class="pre">PARTUUID</span></code> that is needed by <code class="docutils literal notranslate"><span class="pre">ceph-volume</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ parted --script /dev/sdd mkpart primary 1 100%
$ blkid /dev/sdd1
/dev/sdd1: PARTLABEL=&quot;primary&quot; PARTUUID=&quot;16399d72-1e1f-467d-96ee-6fe371a7d0d4&quot;
</pre></div>
</div>
</div>
<div class="section" id="osd">
<span id="ceph-volume-lvm-existing-osds"></span><h2>对于已有 OSD<a class="headerlink" href="#osd" title="Permalink to this headline">¶</a></h2>
<p>For existing clusters that want to use this new system and have OSDs that are
already running there are a few things to take into account:</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>this process will forcefully format the data device, destroying
existing data, if any.</p>
</div>
<ul>
<li><p>OSD paths should follow this convention:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">osd</span><span class="o">/&lt;</span><span class="n">cluster</span> <span class="n">name</span><span class="o">&gt;-&lt;</span><span class="n">osd</span> <span class="nb">id</span><span class="o">&gt;</span>
</pre></div>
</div>
</li>
<li><p>Preferably, no other mechanisms to mount the volume should exist, and should
be removed (like fstab mount points)</p></li>
</ul>
<p>The one time process for an existing OSD, with an ID of 0 and using
a <code class="docutils literal notranslate"><span class="pre">&quot;ceph&quot;</span></code> cluster name would look like (the following command will <strong>destroy
any data</strong> in the OSD):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">filestore</span> <span class="o">--</span><span class="n">osd</span><span class="o">-</span><span class="nb">id</span> <span class="mi">0</span> <span class="o">--</span><span class="n">osd</span><span class="o">-</span><span class="n">fsid</span> <span class="n">E3D291C1</span><span class="o">-</span><span class="n">E7BF</span><span class="o">-</span><span class="mi">4984</span><span class="o">-</span><span class="mi">9794</span><span class="o">-</span><span class="n">B60D9FA139CB</span>
</pre></div>
</div>
<p>The command line tool will not contact the monitor to generate an OSD ID and
will format the LVM device in addition to storing the metadata on it so that it
can be started later (for detailed metadata description see
<a class="reference internal" href="../../../dev/ceph-volume/lvm/#ceph-volume-lvm-tags"><span class="std std-ref">元数据</span></a>).</p>
</div>
<div class="section" id="crush">
<h2>CRUSH 设备类<a class="headerlink" href="#crush" title="Permalink to this headline">¶</a></h2>
<p>要设置 OSD 所属的 CRUSH 设备类，用 <code class="docutils literal notranslate"><span class="pre">--crush-device-class</span></code>
选项。对基于 bluestore 和 filestore 的 OSD 都适用：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span><span class="o">-</span><span class="n">volume</span> <span class="n">lvm</span> <span class="n">prepare</span> <span class="o">--</span><span class="n">bluestore</span> <span class="o">--</span><span class="n">data</span> <span class="n">vg</span><span class="o">/</span><span class="n">lv</span> <span class="o">--</span><span class="n">crush</span><span class="o">-</span><span class="n">device</span><span class="o">-</span><span class="k">class</span> <span class="nc">foo</span>
</pre></div>
</div>
</div>
<div class="section" id="multipath">
<span id="ceph-volume-lvm-multipath"></span><h2><code class="docutils literal notranslate"><span class="pre">multipath</span></code> 支持<a class="headerlink" href="#multipath" title="Permalink to this headline">¶</a></h2>
<p>Devices that come from <code class="docutils literal notranslate"><span class="pre">multipath</span></code> are not supported as-is. The tool will
refuse to consume a raw multipath device and will report a message like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">--&gt;</span>  <span class="ne">RuntimeError</span><span class="p">:</span> <span class="n">Cannot</span> <span class="n">use</span> <span class="n">device</span> <span class="p">(</span><span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">mapper</span><span class="o">/&lt;</span><span class="n">name</span><span class="o">&gt;</span><span class="p">)</span><span class="o">.</span> <span class="n">A</span> <span class="n">vg</span><span class="o">/</span><span class="n">lv</span> <span class="n">path</span> <span class="ow">or</span> <span class="n">an</span> <span class="n">existing</span> <span class="n">device</span> <span class="ow">is</span> <span class="n">needed</span>
</pre></div>
</div>
<p>The reason for not supporting multipath is that depending on the type of the
multipath setup, if using an active/passive array as the underlying physical
devices, filters are required in <code class="docutils literal notranslate"><span class="pre">lvm.conf</span></code> to exclude the disks that are part of
those underlying devices.</p>
<p>It is unfeasible for ceph-volume to understand what type of configuration is
needed for LVM to be able to work in various different multipath scenarios. The
functionality to create the LV for you is merely a (naive) convenience,
anything that involves different settings or configuration must be provided by
a config management system which can then provide VGs and LVs for ceph-volume
to consume.</p>
<p>This situation will only arise when trying to use the ceph-volume functionality
that creates a volume group and logical volume from a device. If a multipath
device is already a logical volume it <em>should</em> work, given that the LVM
configuration is done correctly to avoid issues.</p>
</div>
<div class="section" id="id2">
<h2>存入元数据<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h2>
<p>不管是什么类型的卷（日志或数据）或 OSD 对象存储器，下面的标签都会在准备过程中打上：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">cluster_fsid</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">encrypted</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">osd_fsid</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">osd_id</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">crush_device_class</span></code></p></li>
</ul>
<p>如果是 <a class="reference internal" href="../../../glossary/#term-filestore"><span class="xref std std-term">filestore</span></a> 会加上这些标签：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">journal_device</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">journal_uuid</span></code></p></li>
</ul>
<p>如果是 <a class="reference internal" href="../../../glossary/#term-bluestore"><span class="xref std std-term">bluestore</span></a> 会加上这些标签：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">block_device</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">block_uuid</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">db_device</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">db_uuid</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">wal_device</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">wal_uuid</span></code></p></li>
</ul>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>完整的 LVM 标签惯例见 <a class="reference internal" href="../../../dev/ceph-volume/lvm/#ceph-volume-lvm-tag-api"><span class="std std-ref">Tag API</span></a> 。</p>
</div>
</div>
<div class="section" id="id3">
<h2>总结<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h2>
<p>To recap the <code class="docutils literal notranslate"><span class="pre">prepare</span></code> process for <a class="reference internal" href="../../../glossary/#term-bluestore"><span class="xref std std-term">bluestore</span></a>:</p>
<ol class="arabic simple">
<li><p>Accepts raw physical devices, partitions on physical devices or logical volumes as arguments.</p></li>
<li><p>Creates logical volumes on any raw physical devices.</p></li>
<li><p>Generate a UUID for the OSD</p></li>
<li><p>Ask the monitor get an OSD ID reusing the generated UUID</p></li>
<li><p>OSD data directory is created on a tmpfs mount.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">block</span></code>, <code class="docutils literal notranslate"><span class="pre">block.wal</span></code>, and <code class="docutils literal notranslate"><span class="pre">block.db</span></code> are symlinked if defined.</p></li>
<li><p>monmap is fetched for activation</p></li>
<li><p>Data directory is populated by <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code></p></li>
<li><p>Logical Volumes are assigned all the Ceph metadata using lvm tags</p></li>
</ol>
<p>And the <code class="docutils literal notranslate"><span class="pre">prepare</span></code> process for <a class="reference internal" href="../../../glossary/#term-filestore"><span class="xref std std-term">filestore</span></a>:</p>
<ol class="arabic simple">
<li><p>Accepts raw physical devices, partitions on physical devices or logical volumes as arguments.</p></li>
<li><p>Generate a UUID for the OSD</p></li>
<li><p>Ask the monitor get an OSD ID reusing the generated UUID</p></li>
<li><p>OSD data directory is created and data volume mounted</p></li>
<li><p>Journal is symlinked from data volume to journal location</p></li>
<li><p>monmap is fetched for activation</p></li>
<li><p>devices is mounted and data directory is populated by <code class="docutils literal notranslate"><span class="pre">ceph-osd</span></code></p></li>
<li><p>data and journal volumes are assigned all the Ceph metadata using lvm tags</p></li>
</ol>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../create/" class="btn btn-neutral float-right" title="create" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../encryption/" class="btn btn-neutral float-left" title="加密" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>